Towards sociable virtual humans : multimodal recognition of human input and behavior

One of the biggest obstacles for constructing effective sociable virtual humans lies in the failure of machines to recognize the desires, feelings and intentions of the human user. Virtual humans lack the ability to fully understand and decode the communication signals human users emit when communicating with each other. This article describes our research in overcoming this problem by developing senses for the virtual humans which enables them to hear and understand human speech, localize the human user in front of the display system, recognize hand postures and to recognize the emotional state of the human user by classifying facial expression. We report on the methods needed to perform these tasks in real-time and conclude with an outlook on promising research issues of the future

Modeling and Animating Virtual Humans for Real-Time Applications

International audienceWe report on the workflow for the creation ofrealistic virtual anthropomorphic characters. 3D-models ofhuman heads have been reconstructed from real people byfollowing a structured light approach to 3D-reconstruction. Wedescribe how these high-resolution models have been simplifiedand articulated with blend shape and mesh skinning techniques toensure real-time animation. The full-body models have beencreated manually based on photographs. We present a system forcapturing whole body motions, including the fingers, based on anoptical motion capture system with 6 DOF rigid bodies andcybergloves. The motion capture data was processed in onesystem, mapped to a virtual character and visualized in real-time.We developed tools and methods for quick post processing. Todemonstrate the viability of our system, we captured a libraryconsisting of more than 90 gestures

Interest based selection of user generated content for rich multimedia services

In view of the overwhelming popularity of user generated content, both in terms of production and consumption, new intelligent services are needed to help users finding the content they need and enhance existing services with suitably selected content. In this paper we present a set of algorithms for retrieving content, based on dynamic user profiles and learning capabilities (e.g. based on user feedback). The profile information is used in content searches as well as for assisting the user input analysis process (i.e. speech recognition). To illustrate the approach taken, a rich communication service is presented. Here, the basic service (i.e. voice/video conferencing) is enhanced by showing pictures in real time to the users based on the topic of their conversation and their specific interests